Skip to content

Making triton a soft dependency #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
AnishPahilajani wants to merge 1 commit into from
Closed

Making triton a soft dependency #75

AnishPahilajani wants to merge 1 commit into from

Conversation

@AnishPahilajani
Copy link

@AnishPahilajani AnishPahilajani commented Mar 11, 2025

Making triton soft dependency so we can run GPTQ models

Added a try and except block on 2 imports

Related issue number

How to verify the PR

Was the PR tested

  • I have added >=1 unit test(s) for every new method I have added.
  • I have ensured all unit tests pass

@AnishPahilajani AnishPahilajani changed the title made triton a soft dependency Making triton a soft dependency Mar 11, 2025
@BrandonGroth
Copy link
Collaborator

BrandonGroth commented Mar 11, 2025 •
edited
Loading

Triton is in the pyproject.toml as a hard dependency for hardware-independent matrix multiplication kernels. Do you have any example of how GPTQ was blocked?

@AnishPahilajani
Copy link
Author

AnishPahilajani commented Mar 12, 2025 •
edited
Loading

Yes,
When running inference on Power/Spyre stack I get the following error

Failed to import addon packages: No module named 'triton'
Traceback (most recent call last):
  File "/tmp/aiu-fms-testing-utils/scripts/inference.py", line 211, in <module>
    raise Exception("GPTQ not enabled")
Exception: GPTQ not enabled

I dont think it matters what parameters I used, but here are the parameters:

  • 1 AIU card
  • input = 64 tokens
  • output = 64 tokens
  • batch_size = 1
  • running the granite gptq instruct model
  • gptq is enabled

@BrandonGroth
Copy link
Collaborator

BrandonGroth commented Mar 12, 2025 •
edited
Loading

What is the status of installing Triton on Power? If it doesn't work, we can make Triton an optional package. We already have a framework for doing so.

@AnishPahilajani
Copy link
Author

AnishPahilajani commented Mar 12, 2025

When I try to install using pip.
pip install triton

I get the following error

ERROR: Could not find a version that satisfies the requirement triton (from versions: none)
ERROR: No matching distribution found for triton

@chichun-charlie-liu chichun-charlie-liu mentioned this pull request Mar 20, 2025
Merged
2 tasks
@chichun-charlie-liu
Copy link
Collaborator

chichun-charlie-liu commented Mar 20, 2025

replaced by #78

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrea-fasoli andrea-fasoli Awaiting requested review from andrea-fasoli andrea-fasoli is a code owner

@chichun-charlie-liu chichun-charlie-liu Awaiting requested review from chichun-charlie-liu chichun-charlie-liu is a code owner

@kcirred kcirred Awaiting requested review from kcirred kcirred is a code owner

@nwang-ibm nwang-ibm Awaiting requested review from nwang-ibm nwang-ibm is a code owner

@tharapalanivel tharapalanivel Awaiting requested review from tharapalanivel tharapalanivel is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@AnishPahilajani @BrandonGroth @chichun-charlie-liu